© 




Europaisct^^^tentamt 

European Patent Office 
Office europtfen des brevets 



© Publication number: 



0 257 934 

A2 



@ Application number: 87307190.6 
<§) Date of filing: 14.08.87 



EUROPEAN PATENT APPLICATION 

© int. CI. 4 : G06F 15/20 



© Priority: 22.08.86 US 899109 


© Applicant: AMERICAN TELEPHONE AND 


TELEGRAPH COMPANY 


© Date of publication of application: 


550 Madison Avenue 


02.03.88 Bulletin 88/09 


New York, NY 10022(US) 


© Designated Contracting States: 


© Inventor: Lagarias, Jeffrey Clark 


BEDEFRGBNLSE 


133 Summit Avenue, 24 




Summit New Jersey 07901 (US) 




© Representative: Watts, Christopher Malcolm 




Kelway, Dr. et al 




Western Electric Company Limited 5, 




Mornington Road 




Woodford Green Essex, IG8 OTU(GB) 



© A method and apparatus for optimizing system operational parameters through projective 
transformations. 



© Method and apparatus for optimizing the operational state of a system employing iterative steps that 
approximately follow a projective scaling trajectory or an affine scaling trajectory, or curve (41 , 35, 34, 33,) in 
computing from its present state, x o to a next state *i toward the optimum state. The movement is made in a 
transformed space where the present (transformed) state of the system is at the center of the space (41 ), and 
the curve approximation is in the form of a power series in the step size. The process thus develops a sequence 
of tentative states xt, xi x„.... It halts when a selected suitable stopping criterion is satisfied, and assigns the 
most recent tentative state" as the optimized operating state of the system (28). 
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A METHOD AND APPARATUS FOR OPTIMIZING SYSTEM OPERATIONAL PARAMETERS THROUGH 

PROJECTIVE TRANSFORMATIONS 



Technical Held " 

This invention relates to systems for optimizing processes such as the process of resource allocation. 
5 More particularly, this invention relates to apparatus and methods for optimizing processes in technological 

and industrial environments to minimize the costs or maximize the benefits of such processes. ,8 



Background of the invention 
io ^ 

The need for optimization of systems arises in a broad range of technological and industrial areas. 
Examples of such a need include the assignment of transmission facilities in telephone transmission 
systems, oil tanker scheduling, control of the product mix in a factory, deployment of industrial equipment, 
inventory control, and others. In these examples a plurality of essentially like parameters are controlled to 

75 achieve an optimum behavior or result. Sometimes, the parameters controlling the behavior of a system 
have many different characteristics but their effect is the same; to wit they combine to define the behavior 
of the system. An example of that is the airline scheduling task. Not only must one take account of such 
matters as aircraft, crew, and fuel availability at particular airports, but it is also desirable to account for 
different costs at different locations, the permissible routes, desirable route patterns, arrival and departures 

20 time considerations via-a-vis one's own airline and competitor airlines, the prevailing travel patterns to and 
from different cities, etc. Two common denominators of all of these applications is the existence of many 
parameters or variables that can be controlled, and the presence of an objective -to select values for the 
variables so that, in combination, an optimum result is achieved. 

The relationships describing the permissible values of the various variables and their relationship to 

25 each other form a set of constraint relationships. Optimization decisions are typically subject to constraints. 
Resources, for example, are always limited in overall availability and, sometimes, the usefulness of a 
particular resource in a specific application is limited. The challenge, then, is to select values of the 
parameters of the system so as to satisfy all of the constraints and concurrently optimize its behavior, i.e., 
bring the level of "goodness" of the objective function to its maximum attainable level. Stated in other 

30 words, given a system where resources are limited, the objective is to allocate resources in such a manner 
so as to optimize the system's performance. 

One method of characterizing optimization tasks is via the linear programming model. Such a model 
consists of a set of linear equalities and inequalities that represent the quantitative relationships between the 
various possible systems parameters, their constraints, and their costs (or benefits). 

35 Some optimization tasks cannot be represented by such systems of linear relationships. They involve 
higher powers of the unknowns or other nonlinearities in the relationships and hence are not encompassed 
by the linear programming model. 

It should be noted that the optimization tasks discussed above pertain to real physical problems for 
which businessmen require solutions. It may also be noted that it is not unusual for a physical problem to 

40 be represented by mathematical expressions from which parameter values can be specified for use in the 
physical world to construct or operate a physical system. Typical prior art examples of the use of 
mathematical models to characterize physical systems are the construction of complex filters, design and 
characterization of radio antennas, and control of rubber-molding operations. 

At one time, artisans were unable to explicitly solve any of the optimization tasks that were facing them. 

45 To compensate for that inability, people used intuition and experience to arrive at what they felt was a 
preferred assignment of parameter values. Businessmen who were good at it prospered, and those who 
were not good at it failed. More recently, quantitative tools have become available to assist businessmen in 
these decision-making activities. For example, manufacturing plants use linear programming models to 
control production schedules and inventory levels that will satisfy sales demands and, at the same time, 

so minimize production and inventory costs. Similarly, the AT&T communication network uses linear program- 
ming models to route telephone traffic over a network of transmission facilities so that the entire traffic 
demand is satisfied, transmission costs are minimized, and at the same time, no transmission links are 
overloaded. 
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The best known prior art approach to solving allocation problems posed as linear programming models 
is known as the simplex method. It was invented by George B. Darttzig in 1947, and described in Linear 
Programming and Extensions , by George B. Dantzig, Princeton University Press, Princeton, New Jersey, 
1963. In the simplex method, the first step is to select an initial feasible allocation as a starting point. The 

5 simplex method gives a particular method for identifying successive new allocations, where each new 
allocation improves the objective function compared to the immediately previous identified allocation, and 
the process is repeated until the identified allocation can no longer be improved. 

The operation of the simplex method can be illustrated diagrammatically. In two-dimensionaJ systems 
the solutions of a set of linear constraint relationships are given by a polygon of feasible solutions. In a 

w three-dimensional problem, linear constraint relationships form a three dimensional polytope of feasible 
solutions. As may be expected, optimization tasks with more than three variables form higher dimensional 
polytopes. FIG. 1 depicts a polytope contained within a multi-dimensional hyperspace (the representation is 
actually shown in three dimensions for lack of means to represent higher dimensions). It has a plurality of 
facets, such as facet 11, and each of the facets is a graphical representation of a portion of one of the 

75 constraint relationships in the formal linear programming model. That is, each linear constraint defines a 
hyperplane in the multi-dimensional space of polytope 10, and a portion of that plane forms a facet of 
polytope 10. Polytope 10 is convex, in the sense that a line joining any two points of polytope 10 lies within 
or on the surface of the polytope. 

It is well known that there exists a solution of a linear programming model which maximizes (or 

20 minimizes) an objective function, and that the solution lies at a vertex of polytope 10. The strategy of the 
simplex method is to successively identify from each vertex the adjacent vertices of polytope 10, and select 
each new vertex (each representing a new feasible solution of the optimization task under consideration) so 
as to bring the feasible solution closer, as measured by the objective function, to the optimum point 21 . In 
FIG. 1, the simplex method might first identify vertex 12 and then move in a path 13 from vertex to vertex 

25 (14 through 20) until arriving at the optimum point 21. 

The simplex method is thus constrained to move on the surface of polytope 10 from one vertex of 
polytope 10 to an adjacent vertex along an edge. In linear programming problems involving thousands, 
hundreds of thousands, or even millions of variables, the number of vertices on the polytope increases 
correspondingly, and so does the length of path 13. Moreover, there are socalled "worst case" problems 

30 where the topology of the polytope is such that a substantial fraction of the vertices must be traversed to 
reach the optimum vertex. 

As a result of these and other factors, the average computation time needed to solve a linear 
programming model by the simplex method appears to grow at least proportionally to the square of the 
number of constraints in the model. For even moderately-sized allocation problems, this time is often so 

35 large that using the simplex method is simply not practical. This occurs, for example, where the constraints 
change before an optimum allocation can be computed, or the computation facilities necessary to optimize 
allocations using the model are simply not available .at a reasonable cost Optimum allocations could not 
generally be made in "real time" (i.e., sufficiently fast) to provide more or less continuous control of an 
ongoing process, system or apparatus. 

40 To overcome the computational difficulties in the above and other methods, N.K. Karmarkar invented a 
new method, and apparatus for carrying out his method, that substantially improves the process of resource 
allocation. In accordance with Karmarkar's method, which is disclosed in U.S. patent application No. 
725,342 filed April 19, 1985, a starting feasible solution is selected within polytope 10, and a series of 
moves are made in the direction that, locally, points in the direction of greatest change toward the optimum 

46 vertex of the polytope. A step of computable size is then taken in that direction, and the process repeats 
until a point is reached that is close enough to the desired optimum point to permit identification of the 
optimum point 

Describing the Karmarkar invention more specifically, a point in the interior of polytope 10 is used as 
the starting point. Using a change of variables which preserves linearity and convexity, the variables in the 

so linear programming model are transformed so that the starting point is substantially at the center of the 
transformed polytope and all of the facets are more or less equidistant from the center. The objective 
function is also transformed. The next point is selected by moving in the direction of steepest change in the 
transformed objective function by a distance (in a straight line) constrained by the boundaries of the 
polytope (to avoid leaving the polytope interior). Finally, an inverse transformation is performed on the new 

55 allocation point to return that point to the original variables, i.e.. to the space of the original polytope. Using 
the transformed new point as a new starting point, the entire process is repeated. 
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. Karmarker describes two related "reseating" transformations for moving a point to the center of the 
poiytope. The first method uses a projective transformation, and the second method uses an affine 
transformation. These lead to closely related procedures, which we call projective scaling and affine scaling , 
respectively. The projective scaling procedure is described in detail in N.K. Karmarkar's paper, "A New 

5 Polynomial Time Algorithm for Linear Programming", Combinatorica , Vol. 4, No. 4, 1934, pp. 373-395, and 
the affine scaling method is described in the aforementioned N. Karmarkar '342 application and in that of 
Vanderbei, filed April 11, 1986 and bearing the serial number 851,120. 

* The advantages of the Karmarkar invention derive primarily from the fact that each step is radial within 
the poiytope rather than circumferential on the poiytope surface and, therefore, many fewer steps are 

w necessary to converge on the optimum point. 

Still, even in Karmarkar's method and apparatus a substantial number of iterations are required when 
the number of variables to be controlled is very large. Also, the calculations that need to be performed are 
not trivial because they involve transformations of very large matrices. Fortunately, in most applications the 
matrices are rather sparse because most of the coefficients of most of the variables are zero. This sparsity 

75 permits simplifications to be made which result in substantial improvements in performing resource 
allocation assignments with Karmarkar's apparatus and method. 

The number of iterations that is required in Karmarkar's method is related to the number of variables 
and to the chosen size for each step. But because the step taken is a straight line which only locally points 
in the direction of steepest change in the objective function, it is clear that large step sizes may not be 

20 taken for fear of moving away from the shortest path toward the optimum point, and possibly causing the 
process to converge even slower than with smaller steps. 



Summary of the Invention 

25 

We discovered that Karmarkar's formulation of the problem, in effect, describes a vector field which at 
every point within the poiytope defines a best direction for each and every starting point; and our invention 
employs this discovery to advantage.. From each and every starting point within the poiytope there is a 
unique curve from that point to the optimum vertex, that is obtained by following the vector field exactly. 

30 This curve may be specified by a differential equation, and we call the curves obtained by following this 
differential equation P-trajectories. In addition, there is a second vector field, which we call the affine scaling 
vector field, that arises from another linear programming method. Following affine scaling vector field 
exactly yields another set of curves which we call A-trajectories. The invention comprises methods and 
apparatus for approximately following these curves -to an optimum solution of the linear program in a series 

35 of discrete steps. 

More particularly, in one embodiment of our invention iterative steps are taken that approximately follow 
a P-trajectory. The procedure starts from an initial point that corresponds to a feasible state of the system 
to be optimized, xo, and produces a sequence of tentative states xi, x& x„. It halts when a selected 
suitable stopping criterion is satisfied, and selects the most recent tentative state as the operating state of 
40 the system. Still more specifically, at each iteration k, the (k + 1)* iteration is obtained by using a projective 
transformation 4> k that maps the state of the system, x k , to the center e of a transformed linear problem, and 
advancing in the transformed problem to a next point, £ making use of a suitable power series 
approximation to a P-trajectory through eof the transformed problem. 

The power series approximation uses a power series of order £ 2 in at least one step. A /* order power 
45 series approximation for the step taken at the center of the poiytope, e, has the form 
Yff) = e + vtf + ••• + vjtf (1) 

where "the vectors vi, .Z vj are vectors that depend on the constraints of the physical system to be 
optimized, the optimization objective function, and a chosen reparameterization variable. The new state of 
the system, x k +i, is determined by using the inverse projective transformation x*+* - * * '(jfr+ih 
so In addition to the projective transformation realization of our invention, another version of our method 
follows A-trajectories. The method of our invention develops iterative steps to follow a selected A-trajectory 
using suitable power series approximations. Our procedure advances to a state x k +i using a power series 
approximation to an A-trajectory at the current point x k of the problem. 

55 
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Brief Description of the Drawing 

FIG. 1 depicts a hyperspace polytope representing a linear programming problem, and the prior art 
technique of obtaining a solution by traversing along edges of the polytope; 

FIG. 2 illustrates the concept of moving within the polytope from an interior point toward a vertex, in a 
direction generally specified by an objective function; 

FIG. 3 is a general block diagram outlining the method of our invention; 

FIGS. 4-5 illustrate the sequence of steps employed in a projective scaling embodiment of our 
invention; 

FIG. 6 shows the connective relationship of FIGS. 4-5; 

FIGS. 7-8 illustrate the sequence of steps employed in a affine scaling embodiment of our invention; 
FIG. 9 shows the connective relationship of FIGS. 7-8; 

FIG. 10 depicts one hardware embodiment for implementing the steps described in FIGS. 4-6; and 
FIG. 11 depicts one hardware embodiment for implementing the steps described in FIGS. 7-8. 



Detailed Description 

Conceptual Underpinnings of Our Optimization Process 

As discussed above, the behavior of large physical systems is subject to many factors which impose 
various bounds within which the system must operate. These bounds can often be expressed in the form of 
a collection of linear constraint relationships which can be represented in multi-dimensional space by a 
convex polytope, with every point within the polytope and on its surface representing a permissible state of 
the system. The operational state of the system can also be viewed subjectively by characterizing the 
system as being at a "good" state or at a "not so good" state, based on some set of measuring criteria. 
The measuring criteria, or operational criteria, can often be formulated in the form of a linear relationship, 
called the objective function, which can also be defined on the multi-dimensional space. 

Thus, in a two-dimensional system such as shown in FIG. 2, the constraints may form triangle 31 which 
encompasses the set of all feasible operational states of the system. Line segment 29 defines a line of 
constant "goodness" of the system rate, and arrow 30 describes the direction of increased "goodness". It is 
intuitively apparent that when the system of FIG. 2 is at some state within triangle 31 , such as point 41 , 
there is a direction of movement that would bring the system to another point within triangle 31 (i.e., another 
state of the system), that is better. Such a better point might be point 42. It is also apparent that the final 
optimum operating point of the FIG. 2 system, for the particular objective function chosen, is point 28. What 
may not, perhaps, be apparent is that the direction of movement of our optimization process from point 41 
to point 42 is only generally in the direction of arrow 30. In fact, for most of the points within triangle 31 the 
direction of movement of our optimization process is not the direction of arrow 30. It may be observed, for 
example that when moving from point 41 exactly in the direction of arrow 30, point 43 on line 27 is reached. 
One can go no further in the direction of arrow 30 but that point is not an optimum point for the system. In 
light of that, it becomes apparent that the best direction of movement at any one point within triangle 31 is 
in the direction of point 28, rather than the direction of arrow 30. Alas, to know that point 28 must be 
reached is to know the solution itself. 

We discovered, however, that for each objective function, we can define a vector field within the triangle 
(or the polytope, in the general case) that at each point within the polytope -and not just at the center - 
suggests a good direction for moving away from the point and toward the vertex preferred by the objective 
function. Using this vector field, we have found a way to identify a locus of points within the convex 
polytope forming a particular smooth curve, which defines a curve by which the optimum point can be 
reached. This curved path can be regarded as defining the "most direct" path to a vertex of the polytope 
that is the optimum state of the system in accordance with a specified objective function. Since the curve is 
smooth, when sufficient information is available about it, it can be approximated to any degree of accuracy 
by any one of the known approximation techniques, such as by a Taylor series expansion. When that is 
done, an optimum solution of a linear programming problem may be obtained from any feasible state of the 
system in a small number of steps or, perhaps, even in a single step. The latter depends on the complexity 
of the problem at hand or, phrased differently, on the difficulty of approximating accurately the curve 
defining the most direct path to the desired vertex. When computation of a close approximation is 
inadvisable (e.g., takes too much time), an approximation of the curve with fewer terms permits reaching the 
optimum vertex in a small number of steps. 
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Many different power series approximations to the curve can be obtained using different power series 
parameters. 

More generally, one may consider power series approximations to curves obtained by following a vector 
field inside the polytope of feasible solutions which is defined as a steepest descent direction of the 
objective function with respect to any specified Riemannian geometry on the interior of the polytope. The 
affine scaling vector field is a special case of such a generalized steepest descent vector field. 

Projective Scaling Power Series Optimization Process 



The method of our invention calls for the use of a curved path in the hyperspace of the controllable 
parameters of a physical system in order to move from a non-optimum state of the system to an optimum 
state of the system (in accordance with a stated objective function). This curved path is identified and 
traversed in accordance with the steps described below and depicted in the general flow chart of FIG. 3. 

15 More detailed flowcharts are presented in FIGS. 4-5 and 7-8. 

In. FIG. 3, block 200 accepts input from the physical system and represents the input in a form that we 
call a canonical form , to enable efficient processing of the data. This input represents the state of the 
system to be optimized, or more specifically, this input reports on those parameters of the system that 
affect the system performance and that are controllable. 

20 Blocks 210, 220, 230, and 240 form the main iteration loop of our method. For a given tentative state of 
the system, block 210 develops the appropriate power series approximation of the curve to be traversed 
and block 220 determines a new tentative state for the system. Based on a preselected stopping test the 
tentative state of the system is evaluated and if, according to block 230, the test criteria are not met (i.e., 
the system is not sufficiently close to optimum), the data is conditioned for the next iteration in block 240 

25 and fed back to block 210. When it is determined that the tentative state of the system has met the 
stopping criteria applied in block 230, control passes to block 250 where the system is set to its optimized 
operating state. 

The general block diagram of FIG. 3 holds true for both the projective scaling power series method (that 
approximately follows P-trajectories), and the affine scaling power series method (that approximately follows 
30 A-trajectories). The only difference may be in the point at which the main iteration loop is entered. 

With reference to FIGS. 4-5, which describe the projective scaling method, the first step in our method 
(block 100 in FIG. 4) is to accept input information concerning the given attributes of the physical system 
we wish to optimize (controllable parameters, constraints, and objective function) and to transform this 
information into a structure of relationships that defines the task in the aforementioned -canonical form, to 
35 wit: 

minimize: c T x 
Subject to: A x = 0 

?x=n (2) 

x > 0 

with side condition 
45 A e — 0, 

such that c T x is a normalized objective function. An objective function is normalized if c T x£ 0 for all feasible 
solutions x, and c r x = 0 for some feasible solution x. In the above statement of the task, x = fa, xz x„) 
is a vector of the system attributes which, as a whole, describe the state of the system; n is the number of 
such system attributes; c = (cu cz cj is the objective function which minimizes costs, where "cost" is 
whatever adversely affects the performance of the system; c r is the transpose of vector c; 
A - fan, ai2, atff, a mn » is an m by n matrix of constraint coefficients; b = (by, bz b„J is a vector of 
m constraint limits; and e is a vector of all Vs. In addition, the input provides three parameters m, M and £ 
which specify the degree of approximation the method intends to attain. The constant, m, specifies the 
degree of the polynomial approximation to be calculated, and M and e are two stopping criterion parameters 
for which M = 1& and e = 1 0" 8 are typical values used in practice. 
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§ 

Transforming a general statement of the optimization task (e.g., in the form Ax Sb ) to the canonical 
form of equation (2) can be done in accordance with prior art techniques, such as the one described in 
Section 5 of an article by N.ICKarmarkar entitled "A New Polynomial-Time Algorithm for Linear Program- 
ming," Combinatorica 4(4), pp. 373-395, 1984. That technique consists of the addition of slack variables, 
5 adding an artificial variable if necessary to produce a feasible solution, and application of a suitable 
projective transformation. 

With the canonical form of block 100 in place, our starting point is xp-e and we proceed with the step 
described by block 1 10 in FIG. 4, as we set an iteration index, i, to zero. 

Before proceeding with the block 110 description, it may be useful to note that one way to think of our 
io procedure is that there exists a curved space within the polytope, and our task is to take steps in that 
curved space that bring us closer to the optimum operating point of the system. If one can "flatten" that 
space, one can more easily take bigger steps. To that end, we introduce a reparameterization variable p(t) 
such that 

4£&->0 for all t>0, 

dt 

where t is the step size of the iteration, so that the space within the polytope is "flattened" with respect to 
the parameter t. 

The particular choice of p(t) affects the performance of our method. The simplest choice for p(t) may be 
p(t)=t. A more general choice is 

p(ty= e *jt* (3) 
/-* 

and choosing a good set of a coefficients is important. 
30 To assist in choosing a reparameterization function, we develop in block 110 of FIG. 4 a projection 
operator 

/>= (I-A T Q) (4) 
3s and a projected objective function 

c = FXc (5) 

where 

Q= (A A T )~ l A 
X — diag(xi) 

/ is the identity matrix (having 1's on the diagonal and 0*s elsewhere), 
A is the matrix A, with an additional row- of all 1's appended to it, 
the T superscript designates the matrix transpose operation, and 
the -1 superscript designates the matrix inversion operation. 

In the first iteration there is no reason to suppose that the present state the system is, in fact the 
optimum state of the system. However, having computed the projection operation and Its components in 
block 110, it is convenient at this point to test the state of the system (actual or as tentatively reconfigured 
after each iteration) and determine whether to continue with the next iteration. The test is performed in block 
120 by evaluating the inequality 
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Tf = max 



-mm 



where the Z{ and Xj are calculated using 
* u; = Q X c. 



(6) 



(7) 



As stated earlier, M and e are preselected stopping criteria. 

When the inequality (6) is satisfied, the iterative process terminates. The (primal) solution is when given 
by the vector x which, as stated earlier, is composed of the controllable system attributes. The vector w = 
(Via ••• Wm) is an associated vector dual solution, and w m -x T c is the duality gap , which measures how 
close the objective function value for the solution obtained is to the optimal objective function value. When 
the inequality is satisfied, the values of x are communicated to block 150, as described hereinafter. When 
the inequality (6) is not satisfied, we proceed to compute p(t) and the transformed power series 
approximation ylfcp) in block 130, where 



25 



and, 



30 



y(t;p) - c + £ v k ( P ) t k 



(8) 



(«) 



35 



The a coefficients may be computed in a number of ways. Two ways that yield good results are described 
below. Both develop the a coefficients for p(t) and the v k (p) coefficients for y(t,p), recursively. 
In accordance with the first technique, the recursive process begins by initializing 



40 



45 



V 0 (p) - A 0 (p) - I, 
a x - {e^P c)/(c c) 

Ai(p)= -V x {p)> 



(10) 



Then, for 1 < k Zm, the v and a coefficients are computed by 



so 
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a k - - I T P (*£ Vj(p) ±k-j{p) e), 

v k {p) - -P ( k £vfo)* t -fy)*) + ate (11) 

— <f«'a0 and 
— i-i - 

w A k (p) £ Ay(p)V i _ J (p) 

y-o 

In the above recursive solution of pft). the formula being solved is 

—^(c T Px( p (t)))= t (12) 



as a power series in t. 

20 Next we derive a second technique based on a different parameterization, in this parameterization we 
choose the coefficients a/ in such a way that 
ejx(p(t))mt (13) 

where ei is a vector of all zeros except a 1 in the /"> coordinate position. The coordinate, i, is chosen by 
selecting a component in c . such that c / is the largest positive entry of jc^ , (choosing the smallest i in 
25 case 0 f ties). p or example, if c_ = (4, -2,5.5,1) then a proper choice would be / » 3. With i chosen, the 
recursion for making equation (13) hold is 

<*1 - el* (14) 

30 and 

a k - - <x x tf P (*£ Vj(p) A w (p) e). (15) 
- — i-i— ~ 

35 The vrfp) coefficients are calculated using this value of a k as before, using equation (11). 

Having determined the a and v coefficients, what is left is to select a step size. For p(t) structured as 
above, a number of step selection methods may be used. 

One very simple method is to first compute a trial tentative allocation, for a chosen large step, such as 
t=0.95. If the trial allocation is feasible then the new tentative state of the system may be equated with the 
40 computed trial allocation. If not, then lower values for t may be chosen, such as t = 0.95, 0.9, 0.8, 0.7,..., 0.1 
and the feasibility determination repeated until a feasible point is found. We expect that a feasible point 
would almost certainly be found before t = 0.2 and nearly always for t = 0.95. 

A second method for determining a step size is to minimize the -potential function" of Karmarkar, 
defined by 



9c{y) - nlog ( e T y ) - £ log ( a* ) (16) 



on the line segment connecting e (the starting point of the iteration) and £1 (the trial tentative allocation), to 
obtain a point jji. This may be done by "line search," using standard prior art methods. Noting that £1 might 
be outside the potytbpe, we restrict the line search to that part of the line segment where all components yi 
are greater than 0. 

M A third method is to do a "line search" along the curve, Yft,pj on the interval 0 £ t £ 1 to minimize 
Karmarkar's potential function and to choose # to be the minimizing point 
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In FIG. 5 f block 140 computes the step size in accordance with the first method. The initial step size 
taken Is t=0.95, a trial tentative allocation, /1 is computed for this t f and the allocation is evaluated with 
respect to its feasibility (i.e., it is queried whether all components of / > 0). The value of t is reduced as 
necessary, until a strictly feasible new tentative allocation gi is developed. 

Once a feasible new tentative allocation is determined, a transformation of the determined state of the 
system is necessary in order to bring the system back to its original coordinates. This is accomplished in 
block 140 with the projective transformation 

= (nXy/e T Xy). (17) 

The index i is incremented, and control passes to block 110 for the next iteration. 

Affine Scaling Power Series Optimization Process 

The above description illustrates a projective transformation method. Now we describe an affine scaling 
power series method, which approximately follows the somewhat different A-trajectories. It is very similar to 
the projective transformation method. This method produces a sequence of iterates starting from an 
initial point £o; and is described in the flowchart of FIGS. 7-8. 

In FIG. 7, blocks 310, 320 and 330 serve the function of block 200 in FIG. 3. In block 310, the input 
linear program is required to be in standard dual form , which is: 

maximize b T y 

subject to A < c (18) 



where A is m x n matrix, b and £ are m x 1 vectors, and c is an n x 1 vector. Such a representation can be 
30 obtained as the dual to the linear program given in the standard form of equation (2). By adding a vector of 
non-negative slack variables vv, the above may be rewritten as: 



35 



maximize b T y 

subject to A^y + w — c , (19) 
w > 0. 



40 where w is an m x 1 vector. The affine scaling differential equation for an A-trajectory is 

dy{t) /dt = p{t) [a Mf~ 2 A T ]~\6 f (20) 

45 

y(0) — y Q , 

where 

so w — w(t;p) mm c — A T j/(t;p)j 

W - W(t;p) - diag(w) 9 (21) 

and 

55 P{t) - E (22) 

i-o 
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is a power series used to reparameterize the A-trajectory. 

The following describes a method giving one particular choice of p(t), though other choices of p(t) 
could be used. This choice of p(t) is defined as follows. Starting at the current feasible point y k , which we 
will denote jf, let vv* = (w* w*)be defined by 
y£ = c -A T i*. (23) 1 m 

The power series parameterization is chosen so that w(tp(t) ■ (m(tp(t)),..., w m (tp(t))) satisfies 
wrfMt)) (24) 

where i G is that coordinate index of the objective function that is most rapidly increasing. Specifically, io is 
the smallest index that minimizes the function 

«,„ i^k (25) - 

^(w^ 2 A T r i bj 

over all i such that 

where W* =* diag(w*). 

Selection of the coordinate index is depicted in block 320 in FIG. 5 

The idea behind using this particular reparameterization is that a linear approximation to the power 
series expression for tft;p) becomes infeasible at t - 1 , so we expect a step size to be on the order of t = 1 . 
The test embodied in equation (25) is a ratio test applied to a first order approximation to w(t;p). 

With the above formulation, the following recursion may be used to compute the power series 
expansions: 

oo 

y(*;p) = (26) 

" *-o — 

and 



WW 

w(tlP) - £ w k **• (27) 
— *-o — 

We initialize the recursion as follows. Our initial data is 
vo « & (28) 
from which wp is computed by: 
wp = c -A T vo (29) 
Now define auxiliary quantities {X*} by 

x(t) = J^- 2 (0= E o £*«* (30) 

60 where each X k is a diagonal matrix, and initialize with 

Wo = diag(wp) 

£o = W~*. (31) 

It is also useful to introduce that auxiliary matrix quantities {M k } defined by 



M(t) = AW{t)' 2 A T - J] Mk t h (32) 
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so that 

M k = AX k A T (33) 
and 

M 0 =£Xp£ r . (34) 
5 Next, compute 

Po = (Mpb)f> (35) 
and then initialize 
ao ■ WV/o (36) 

where is the /oth coordinate of w 0 , and /o is the index determined in block 320. Lastly, we set 
10 v = ao M 7 1 b. (37) 

All of the above initializations are done in block 330 in FIG. 7. 

Following the initializations, the process of FIGS. 7-8 proceeds with a recursion step to develop the 

coefficients of the power series that describes the path along which the process moves towards its optimum 

state. At the beginning of the (f + if* step the value up to and including all coefficients of Vj is K nown » ( as 
75 well as all w?, X h M { , a, for 0 £ i Z The values of wj, $ Mj t a } and y + 1, are then computed in block 

340, as follows. 



20 



2S 



UN 



* C — A Vj 

Wj — diag(wj) 

- — U-o- 

Mj » A XjA T . 



/-o 



, and 



(38) 



Next, in block 350, v / + 1 is computed using 



so 



35 



— 1 



AfZ 1 



j+l — 

and a/ is determined by solving 

ay - -(i+l) 0o l («y+i)*o- 



(39) 



(40) 



40 



Lastly, in block 360, the value 



cry j 



(41) 



is computed to complete the recursion step before entering the conditional block 370. The recursion is 
45 repeated for as many terms (m in FIG. 7) in the power series as is desired. 

The computation of X/ given by (38) may be simplified by computing, at the fih step for (for / £ 1) 
auxiliary quantities 



so 



cry- s Wi 

— t-o 



(42) 



55 



and saving all values (Uf 1 ZiS /?. The the calculation of Xj in equations (38) is replaced by 



s 1 x k Uj.A 

*-o J 



(43) 
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Having computed ttie coefficients for the power series, 

E ***** (44) 

one needs to select a step size, t. As before, we chose t = .95, .9, .... 1 and check in block 380, for each t, 
whether w(t) is strictly feasible , i.e., whether w(t) all entries are positive. The first of the values of t tested 
for which this holds true is selected, and the new vector 
Yr + i -tfj (45) 

is defined to be our next tentative state of the system, and is the starting point for the next iteration. 

To determine whether a next iteration is necessary, the newly developed tentative state is tested in 
block|390. As a stopping rule we use the ratio test 



\c T y n — c T y n+1 | 



c T y»-il 



<e (46) 



20 where € is a stopping criterion value given as ain input quantity. When the computed ratio test is greater 
than c, control passes to block 330 and a new iteration is initiated. When the ratio is less than we 
conclude that the developed tentative state, y, corresponds closely to the optimum vertex in the polytope of 
FIG. 1 , and we conclude that the desired state of the system has been reached. 



25 

Illustrative Apparatus Embodying Our Invention 



Implementation of our process in hardware is most easily accomplished by directing attention to the 
blocks in, for example, FIGS. 4-5, and implementing each block in a hardware module. The hardware blocks 

30 may be physically different and separate elements, or they may be combined to reside in one physical 
machine. FIG. 10 depicts a system which employs a different module, or a number of modules, for the 
different blocks in FIGS. 4-5. 

The FIG. 10 system is very useful in addressing the prevalent task of resource assignments in various 
enterprises, such as the airline industry or the telecommunications network. The telecommunications 

35 network presents a particularly interesting challenge because it is very large and because there is real and 
substantial financial reward in maintaining the network in an optimum operating state. 

Briefly describing the situation, telecommunication traffic through the United States exhibits large 
geographic variations because of time zone differences, seasonal differences (particularly in vacation spots), 
holiday calling pattern differences, etc. In addition to predictable trends, unusual situations often arise which 

40 also affect the traffic. To best allocate the resources of the communications network so as to most 
effectively serve the public with the available resources, one must deal with many controllable variables. For 
example, telephone trunks of varying capacities are allocated to different types of traffic, traffic is assigned 
and reassigned to different intermediary routes, the work force is administered, etc. In quantitative terms, 
the task of structuring the AT&T communications network so as to place it at an optimum operating state 

45 involves over 1.2 million variables, over 150,000 constraints and more than 100 long-distance switching 
offices. Consideration of the task on this detailed level has not been possible heretofore; but now, in 
accordance with the principles of our invention, the apparatus of FIG. 10 can handle the task in real time to 
automatically adjust the network to the changing traffic conditions. 

In FIG. 10, modules 1200-1230 perform the data collection and conversion steps illustrated by block 

so 200 in FIG. 3 Module 1251 performs the assignment of operating parameters step illustrated by block 250 
in FIG. 3, and the remaining modules in FIG. 10 perform the steps encompassed by the iteration loop of 
FIG. 3. 

Describing FIG. 10 in more detail, blocks 1200-1220 are the data providing elements, and they may be 
elements that presently exist in the telephone network. As an example, element 1200 represents a set of 
55 Traffic Data Recording Systems (TDRS) which are found in many telephone central offices. A TDRS 
measures the usual level of telephone trunks and groups of trunks in the central office and, currently, sends 
this information to a centrally located magnetic tape recorder. Element 1210, represents a set of Force 
Administration Data Systems (FADS) terminals which are used by the telephone companies to assess the 
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level of work force conditions. Element 1220, represents a set of route status sensors that detect overload 
or malfunction conditions in the transmission network. Other sources of data pertaining to the controllable 
parameters of the telephone network may be employed, but for the illustrative purposes of this disclosure 
only elements 1200, 1210 and 1220 are shown in FIG. 10. 

s The data providing elements of FIG. 10 are connected to processor 1230, to which memories 1231 and 
1232 are also connected. Also connected to processor 1230 is an input device 1225 which specifies the 
network constraints and the objective function which is wished to be minimized. As explained earlier, the * 
constraints set the ground rules for permissible states of the telephone network. As an example, some of 
the constraints are of the form of the number of trunks between New York and Boston devoted to data, plus 

70 the number of trunks devoted to TV, plus the number of trunks devoted to voice is less than or equal to a v 
fixed value (corresponding to the total number of trunks between New York and Boston). Information 
concerning the actual division of trunks is provided by the sensors of FIG. 10, whereas the fixed value is a 
constraint supplied by device 1225. The objective function supplied by device -1225 is a statement of the 
desired optimization; such as an enumeration of the different cost and benefit consequences from the use 

75 of different resources of the network. 

Processor 1230 accepts the inputs provided by sensors 1200-1220, structures the data in the canonical 
form described above (and in block 100 in FIG. 4), and stores the structures input in memory 1231 in a 
predetermined manner so as to create the matrices A and _A_. Processor 1230 also communicates the 
objective function c and the current state of the system (as reflected by the controllable parameters 

20 identified by sensors 1200-1220 and augmented by the transformation performed by processor 1230) to 
memory 1232. 

Based on the information available at memories 1231 and 1232 processors 1241, 1242 and 1243 
compute the Z_ vector, the P matrix and the O matrix, respectively, in accordance with the block 110 of 
FIG. 4. These computations may be done with standard mathematical techniques which, of course, form no 

25 part of our invention. Having computed the P andO matrices and the augmented vector c , processor 1250 
is ready to compute the 5 and y signals specified in block 120 of FIG. 4 (receiving the c and x vectors 
needed for these computations from memory 1232). These signals are used in processor 1250 to determine 
whether the stopping criterion has been met. When it is, processor 1250 disables processor 1260 (via line 
1252) and enables processor 1251 (via line 1253); otherwise, processor 1251 is disabled and processor 

30 1260 is enabled. 

Processor 1260 is responsible for recursively computing the values of a coefficients for p(t) and the 
values of v k coefficients, based of P obtained from processor 1242, Z_ obtained from processor 1241, and 
c obtained from memory 1232. The number of recursions, k, within processor 1260 is selected based on 
the- degree of the desired polynomial approximation. The processes, however, are repetitive and straightfor- 

35 ward, as described in block 130 of FIG. 5. 

To complete the iteration, the polynomial approximation derived by processor 1260 is applied to 
processor 1270, where the tentative new state of the system is determined. It is there that a step size is 
selected, a new state is derived for the selected step, and the derived state is evaluated to determine 
whether it is strictly feasible. When the state is found not to be strictly feasible, the step size is reduced and 

40 the process is repeated. When the state of the system is found to be strictly feasible, then it is reprojected 
to place the state of the system in the center of the polytope, and that new tentative state of the system is 
placed in memory 1232. 

Finally, when the stopping criteria are met, control passes to processor 1251 which fetches the latest 
tentative state of the system from memory 1232 and concludes that this is the desired state of the system. 

45 With conventional circuitry, the desired state of the system is either implemented directly, by processor 
1251 communicating switch control commands to the system, or indirectly, by the processor 1251 sending 
messages to personnel who effect the specified changes. In the case of FIG. 10 telephone plant 
optimization system, processor 1251 controls the trunk assignments and routing processors (already in the 
telephone plant) directly, and sends electronic messages to AFADS to schedule and assign the work force 

50 where its needed. 

In the above communications system, scheduling forms merely a part of the optimization task. In many 
other applications scheduling takes a more central role in optimally allocating resources. Optimization of - 
scheduling is a common task in many industries where resources, supplies, facilities and customers are 
diverse and geographically scattered. One example of such an industry is oil refining where there is a 
55 multitude of controllable parameters, constraints, and cost factors. Examples of the above are price of crude 
in different oil-producing countries, on the spot market, and on the various tankers already in transit, 
anticipated arrival time of tankers at different ports, cost of moving the crude to the refineries, capacities 
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and operating constraints of the refineries, import use and other taxes, needs of different customers for 
different refining products, locations qf_ customers vis-a-vis the refineries, distribution network costs, 
inventory controls, etc. The number of controllable variables -which are the scheduled assignments -can 
run into thousands, and more. 

5 In the context of such a scheduling task the method of our invention, represented by the flow chart of 
FIGS. 4-5, can be employed in apparatus such as described by FIG. 10; or as embodied in the flow chart of 
FIGS. 7-8, can be employed in apparatus such as described by FIG. 11. 

For illustrative purposes, FIG. 11 depicts a scheduling/resource allocation optimizer for oil refining 
enterprise. The optimizer employs elements that individually are quite conventional, in combination, to 

10 effectuate the method of our invention. Elements 1100-1120 are sensors that represent the state of the 
enterprise with respect to those parameters that can directly or indirectly be controlled. Element 1100 in 
FIG. 11 represents a set of "inventory information ■ sources which report on the inventory level at the 
different storage facilities of the enterprise. This may include automatic level reading apparatus, computer 
databases in combination with data communication apparatus, etc. Element 1110 represents a set of "raw 

75 materials supply information" sources, and they may be similar in kind to the sources employed by the 
1100 set of sensors. Elements 1110 report, for example, of the available supplies on different tankers at sea 
and at depots of different suppliers, and so forth. Element 1120 is a set of "manufacturing facilities 
information" sources, and they may contain sensors for supplying information concerning the operational 
level (e.g., a percentage of capacity) of each facility. 

20 All signals from the various sensors are applied to processor 1130, but in addition to these signals, 
element 1115 provides information to processor 1130 relating to the operational constraints of the 
enterprise, the cost factors, and the objective function which forms the criterion for measuring performance 
of the enterprise. Processor 1130 selects the necessary slack variables and develops the matrix A as 
specified by equation 19 above. 

25 Processor 1130 stores the A matrix and the b and c vectors of equation 19 in memory 1140, and 
applies, the present (un optimized) state of the enterprise, y , to processor 1150. Processor 1150 selects the 
coordinate /o in accordance with equation 24. The contents of memory 1140 and the output of processor 
1150 are made available to recursion processor 1160. 

Processor 1160 recursively developes the v k coefficients that enable one to compute the next tentative 

30 state of the enterprise whose resources are to be optimally scheduled, first, signals w& Wb, Xo and M 0 are 
derived in processor 1160 in accordance with equations 29-34, from which coefficients /So and ao are 
developed per equations 35-36. The coefficient v^ is then developed as described by equation 37, signals 
wi, Wi, Xi and M\ are developed per equation 38 and a v 1 signal is developed in accordance with equation 
39. From Z 1, ai and vi are developed per equations 40 and 41, bringing processor 1160 to the point where 

35 the next recursion can begin, to develop 02 and vs. When the desired number of coefficients is developed in 
processor 1160, the next tentative state of the enterprise can be determined with the aid of equation 26; 
once the step size, t, is selected. 

The step size selection and the actual determination of the next tentative state is developed in 
processor 1170. As before, the step selection may proceed in a very simple manner by selecting t=0.95 

aq and evaluating the resultant tentative state, ff that state is strictly feasible, then the value of t is accepted, 
the new tentative step is accepted, and a new iteration is initiated with processor 1170 placing the tentative 
state into processor 1150 and enabling processor 1150 to select a new coordinate position, to. When it is 
determined that the resultant state is not strictly feasible, a smaller value of t is selected and a new tentative 
state is computed and evaluated, as before. 

45 Of course, one must also select a test for determining when the tentative state developed is close 
enough to optimum state, enabling one to terminate the scheduling process. This test is made in processor 
1180, and it may follow, for example, the test described in equation 46. Stopping the process is achieved 
with control lead 1181 emanating from processor 1180 and terminating at processor 1260. When the 
optimization process is complete, processor 1180 transfers the desired state of the enterprise to scheduling 

50 communicator 1190 which transmits the various scheduling assignments to the enterprise. 

The above specification of our invention contains a number of specific illustrative embodiments. These 
are intended for explanatory purposes and should not be construed as a limitation on our invention, since 
many changes and modifications can easily be contemplated by those skilled in the art that do not depart 
from the sprit and scope of our invention. 
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Claims 



1 . Apparatus for optimizing operational state of a system described by a set of operational parameters, 
with said optimizing accomplished by adjusting values of said operational parameters within a preselected 

5 set of constraints to minimize a set of preselected operational criterion values, 
CHARACTERIZED BY 

sensors for developing signal representations of said values of said operational parameters; 
a first processor portion, responsive to said signal representations of said sensors, to applied values of said 
constraints and to said operational criterion, for developing a canonical form signal representation of said 
10 operational state of said system, such that said minimizing is effected by 

minimizing: c T x 
Subject to: A x — 0 

75 J* ~" 

c x — n 
x > 0 

with side condition 
20 At — 0, 

where x is a vector related to said operational parameters, £ is a vector related to said set of preselected 
criterion values, n is the number of said operational parameters, A = fan, a& a rt * a mn) is an m by n 
matrix of coefficients related to said preselected set of constraints, and £ is a vector of all 1's; said 
canonical signal representation forming a multidimensional space, with said operational parameters being 
the variables of said space, said matrix defining a polytope in said space, and said c vector specifying a 
direction in said space; 

a second processor portion, responsive to said first processor portion, for projecting said operational 
parameters, said matrix, and said c vector unto a transformed space, and developing a matrix O = 
( AA T r y A > a matrix P = 0-& T Q)* and a vector = P D c, where D - diagfx*); 

a third processor portion, responsive to said second processor portion, for computing a power series 
function in said transformed space of order greater than one that approximates a trajectory curve in 
consonance with said operational criterion; and 

a controller, responsive to said third processor portion, for setting said operational parameters of said 
system at values corresponding to a point in said transformed space and along said curve. 

2. The apparatus of claim 1 
CHARACTERIZED IN THAT 

said third processor portion employs a reparameterization function p(t) in developing said power series 
function such that 

44^>Otor*llt>0 
dt 

3. The apparatus of claim 1 
CHARACTERIZED IN THAT 

said third processor portion employs a reparameterization function p(t) in developing said power series 
function satisfying the equation 
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4. The apparatus of claim 1 

CHARACTERIZED IN THAT — ~ 

said third processor portion employs a reparameterization function p(t) in developing said power series 

function satisfying the equation 

5. The apparatus of claim 1 
CHARACTERIZED IN THAT 

said second processor portion comprises 

a first subprocessor responsive to said first means for developing a matrix 0 » (( AT T )' 1 A ); 
w a second subprocessor responsive to said first means for developing a matrix P = (I - A T ( a A r r ? A ); 
and 

a third subprocessor responsive to said first means for developing a vector c - P Dc. 

6. The apparatus of claim 4 
CHARACTERIZED IN THAT 

ts said controller comprises a first module, responsive to said second processor, for developing an iteration 
stop signal, and a second module responsive to said first module for adjusting said values of said 
operational parameters. 

7. The apparatus of claim 3 
CHARACTERIZED IN THAT 

20 said second processor portion comprises: 

a first subprocessor responsive to said first means for developing a matrix O « (( A A T )" 1 A ); 

a second subprocessor responsive to said first means for developing a matrix P=(l- A T (A A, r J" y A ); 

and 

a third subprocessor responsive to said first means for developing a vector c = P D c. 
25 8. The apparatus of claim 6 
CHARACTERIZED IN THAT 

said controller comprises a first module, responsive to said second processor, for developing an iteration 
stop signal, and a second module responsive to said first module for adjusting said values of said 
operational parameters. 
30 9. The apparatus of claim 7 
CHARACTERIZED IN THAT 

said third processor develops coefficients v k for said power series function, by recursively evaluating 
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Vk — — P( J] VjAk-jC + <*jfcc), where 
— jr— l ~ 

A* - - £ AyVjfe-i 
i-o 

40 V k - diagV h and 



" 



wherer equals c T or equals e T 
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FIG. 5 
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